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ABSTRACT 

Surface-wave  over-the-horizon  radars,  especially  ones  located  in 
tropical  areas,  such  as  Northern  Australia,  are  usually  strongly  af¬ 
fected  by  external  impulsive  noise.  Apart  from  thunderstorm  activ¬ 
ity,  man-made  (industrial)  noise  over  typically  quite  long  coherent- 
integration  time  often  is  of  impulsive  nature  as  well. 

In  this  paper  we  analyse  the  efficiency  of  temporal  and  spatial 
adaptive  techniques  for  impulsive  noise  mitigation.  We  demon¬ 
strate  that  for  heavily  contaminated  dwells,  new  spatio-temporal 
adaptive  processing  is  most  effective.  Initial  impulsive  noise  mit¬ 
igation,  produced  by  adaptive  spatial  processing  is  used  for  range 
and  azimuth  dependent  sea-clutter  spectrum  estimation.  Estimated 
sea-clutter  spectrum  is  then  used  to  “restore”  the  “missing”  data, 
originally  contaminated  by  impulsive  noise. 

1.  DESCRIPTION  AND  ANALYSIS  OF  MITIGATION 
TECHNIQUES 

The  High  Frequency  Over-the-Horizon  Radar  (HF  OTHR)  proba¬ 
bly  constitutes  the  most  prominent  example  of  radars  subjected  to 
severe  impulsive  noise  interference.  Tropical  thunderstorms  which 
are  extremely  active  in  equatorial  regions  such  as  Northern  Aus¬ 
tralia,  typically  generate  a  significant  number  of  lightning  strikes 
within  the  operational  range  of  HF  OTHR  due  to  relatively  long 
coherent  processing  intervals.  In  [2]  based  on  experimental  data 
collected  in  Northern  Australia,  we  introduced  point  process  mod¬ 
els  for  atmospheric  noise  adequate  to  spatial  and  temporal  adap¬ 
tive  impulsive  noise  mitigation.  It  has  been  suggested  that  optimal 
mitigation  technique  should  incorporate  both  spatial  and  temporal 
domains  based  on  the  properties  of  particular  lightning  strike. 

Our  recent  experimental  trial  conducted  from  May  to  Septem¬ 
ber  2000  in  Northern  Australia  revealed  that  accidental  human- 
made  noise  that  quite  often  interferes  with  a  HF  radar,  is  in  most 
cases  also  highly  nonstationary.  The  atmospheric  strike  typically 
occupies  a  single  repetition  period  or  at  most  a  few  consecutive 
repetition  periods  ( for  high  air-mode  waveform  repetition  frequen¬ 
cies  WRF=40  -  60  Hz),  man-made  impulsive  interference  typically 
occupies  significantly  longer  intervals,  measured  in  dozens  of  rep¬ 
etition  periods  (sweeps).  Typical  examples  of  atmospheric  and 
man-made  impulsive  noise  are  presented  in  Fig.  1,  2.  The  am¬ 
plitude  of  the  range  processed  data  at  the  output  of  one  particular 
beam  are  shown  for  different  ranges  (y-axis)  as  a  function  of  repe¬ 


tition  period  (x-axis).  One  can  see  significant  difference  in  number 
of  sweeps  contaminated  by  atmospheric  and  man-made  impulsive 
noise.  Another  important  feature  demonstrated  by  these  figures  is 
the  availability  of  “sea  clutter-free  ranges”.  These  ranges  allow 
for  straight-forward  identification  of  sweeps  affected  by  impulsive 
noise. 

Obviously,  analysis  of  impulsive  noise  mitigation  efficiency 
should  be  expanded  to  man-made  interference.  Indeed,  since  only 
up  to  30%  of  entire  dwell  is  typically  corrupted,  there  is  a  reason 
to  compare  spatial  techniques  with  temporal  ones[l]. 

In  this  paper  we  introduce  comparative  analysis  of  different 
temporal  and  spatial  adaptive  techniques,  suitable  for  impulsive 
noise  mitigation. 

Since  the  actual  interval  corrupted  by  impulsive  noise  is  easily 
identified,  temporal  techniques  are  focused  on  a  proper  estima¬ 
tion  of  the  missing  sea-clutter  data.  For  surface-wave  radars  with 
typically  very  high  sub-clutter  visibility  that  can  range  far  above 
60  dB,  an  accurate  estimation  can  become  a  problem. 

To  address  this  problem  two  major  approaches  could  be  adopted. 
The  first  one  is  based  on  classical  Weiner  prediction  filter.  Compli¬ 
cated  nature  and  range/azimuth  variability  of  sea-clutter  Doppler 
spectrum  impose  limitations  on  the  actual  accuracy  of  this  ap¬ 
proach. 

The  second  technique  is  based  on  direct  optimization  of  re¬ 
placement  data  to  minimize  the  total  power  within  the  specified 
range  of  Doppler  frequencies  which  are  expected  to  be  free  of  sea 
clutter.  This  technique  has  a  different  limitations,  especially  when 
the  number  of  missing  data  is  quite  large  and  consecutive.  How¬ 
ever,  in  attempt  to  minimize  the  overall  power,  strong  targets  could 
be  suppressed  and  some  important  features  of  the  sea-clutter  spec¬ 
trum  could  be  significantly  damaged.  Spatial  techniques  are  effec¬ 
tive  when  strong  impulsive  interference  impinges  on  a  beampat- 
tem  sidelobes.  Meantime,  when  the  entire  coverage  is  important, 
there  would  always  exist  directions  corrupted  by  impulsive  noise 
propagated  via  the  main  beam. 

Comparative  analysis  of  the  above  mentioned  techniques  was 
done  firstly  on  uncorrupted  SW  OTHR  data  with  subclutter  visi¬ 
bility  close  to  the  limit.  One  selected  example  is  shown  in  Fig. 
3.  A  certain  number  of  sweeps  has  been  nominated  as  being  “cor¬ 
rupted”  and  two  abovementioned  temporal  techniques  have  been 
used  to  restore  the  “missing”  data. 

In  order  to  apply  the  classic  prediction  (interpolation)  approach, 
we  first  estimated  the  sea-clutter  temporal  covariance  matrix.  With 
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N  =  1000  repetition  periods  typically  used  in  ship  mode,  we  se¬ 
lected  M  <  N/2,  M  =  400  as  a  dimension  of  prediction/interpolation 
filter  in  expectation  that  whatever  the  actual  number  of  missing 
repetition  periods  is,  there  still  should  be  a  sufficient  number  of 
uncorrupted  repetition  periods  (sweeps)  within  corresponding  M- 
variate  “sliding  window”  of  our  prediction  filter.  The  M-variate 
(range-dependent)  sea-clutter  covariance  matrix  is  estimated  here 
by  forward- backward  averaging: 


n~m+i 

Rd=  (Xf(Xf)B +jX*iXf‘j) 

1 


where 


(1) 


(2) 

(3) 


and  xj  is  the  complex  number  that  corresponds  to  j-th  repe¬ 
tition  period  and  d-th  range  cell.  Particular  beam  number  is  not 
essential  for  this  temporal  processing. 

Let  us  introduce  an  M  x  (M  —  m)  variate  incidence  matrix 
him  that  is  constructed  as  standard  identity  matrix  with  m  deleted 
rows  at  positions  that  correspond  to  the  “missing”  sweeps.  Then 
the  adaptive  prediction  filter  that  generates  an  estimate  of  the  k-th 
missing  data  is  defined  as 

Wt  =  _1  Hmrt  k  =  l,...,m  (4) 

where  ffc  is  k-th  column  of  the  M-variate  matrix  i2j. 

Correspondingly,  the  estimate  k  —  1, . . . ,  m  of  th  k-th 
missing  sweep  is  defined  as 

xi  =  WiHHlXd,  k  =  1 . m  (5) 


Our  second  approach  is  based  on  direct  search  for  the  m-variate 
vector  Xm  for  “missing”  data  that  with  respect  to  the  remaining 
(N  —  m)  “valid”  data  results  in  the  minimal  total  power  within 
some  designated  range  of  Doppler  frequencies. 

Specifically  let  us  present  the  overall  N-variate  vector  Xd  as 

Xd  =  Xt  +  Amxm  (6) 

here  Xd  is  a  N-variate  vector  with  zeroes  the  positions  of  “miss- 
ing”  data,  Am  is  N  x  m-variate  incidence  matrix,  where  rows  of 
the  m-variate  matrix  are  “spread”  over  N  rows,  corresponding  to 
the  positions  of  the  missing  data. 

Weighted  Discrete  Fourier  Transform  (DFT)  over  the  vector 
Xd  could  be  presented  as 

F  D(Xd  +  Amrm)  (7) 

and  with  (N  —  n)  x  N  selection  matrix  S,  the  ( N  —  m)- variate 
vector  of  selected  Doppler  bins  within  the  d-th  range  Doppler  spec¬ 
trum  could  be  presented  as 

SFD(Xd  +  Amzm)  (8) 

where  F  is  the  N-variate  DFT  matrix,  D  is  a  diagonal  weighting 
matrix  (e.g.  Blackman  window). 


Finally,  the  overall  power  within  this  Doppler  window  could 
be  presented  as 

XdHDFHSSTFDXo  +  x%A*DFHSSTDAmxm  +  (9) 

crx”A”DFHSSTFDX$  +  X$H DFH SST FDAmxm.  (10) 

Correspondingly  the  optimum  solution  is 

xm  =  ~{t&DFH  SST  FDAm]~l  A^DFH  SST  DXt  (11) 

(For  rank  deficient  matrix  [A%DFH SST FDAm]  this  solution  is 
modified  to  operate  on  signal  subspace  of  this  matrix.)  Now  these 
techniques  could  be  compared.  Fig  4  presents  the  Doppler  spectra 
for  m  as  100  of  “missing”  data  for  one  range  cut  Both  random 
(atmospheric  like)  and  continuous  (man-made  like)  distributions 
of  “missing”  data  within  the  400  sweeps  long  window  have  been 
analysed.  Different  number  of  missing  sweeps  have  been  analysed, 
m  —  1, 40, 60, 100,  however  only  m  —  100  continuous  case  pro¬ 
cessed  with  optimization  filter  is  shown  (the  only  one  which  shows 
any  difference  from  the  original). 

The  results  demonstrate  that  for  randomly  distributed  “miss¬ 
ing”  data  both  techniques  provide  equally  good  restoration.  The 
prediction  errors  are  equally  small  and  sub-clutter  visibility  is  re¬ 
stored  to  the  original  level  in  this  case.  However  in  the  case  of 
continuous  “missing”  data  both  methods  work  equally  well  only 
for  a  small  number  of  “missing”  sweeps.  For  increased  number  of 
consecutive  missing  data  the  difference  between  these  two  tech¬ 
niques  becomes  more  significant.  While  classical  prediction  is 
still  efficiently  restoring  missing  data  (up  to  100  of  missing  data 
for  400- variate  prediction  filter),  optimization  (11)  generates  esti¬ 
mates  xm  significantly  different  from  the  true  missing  ones.  These 
estimates  lead  to  reduction  in  overall  noise  power  within  the  spec¬ 
ified  Doppler  area,  but  the  overall  structure  of  the  Doppler  spec¬ 
trum  changes  significantly.  For  most  practical  applications  these 
changes  could  not  be  tolerated.  Moreover,  with  significant  num¬ 
ber  of  “degrees  of  freedom”,  total  power  minimization  could  con¬ 
siderably  reduce  the  target  signal  as  well.  Thus,  for  a  randomly 
distributed  missing  data  or  small  number  of  consecutive  missing 
data  (up  to  20  consecutive  sweeps)  the  optimization  technique  (11) 
could  be  recommended  as  a  preferred  option  since  it  does  not  in¬ 
volve  (adaptive)  sea-clutter  spectrum  estimation.  For  typical  man¬ 
made  impulsive  interferences,  this  approach  is  not  appropriate  and 
attention  should  be  attracted  to  a  practical  implementation  of  adap¬ 
tive  prediction  technique  (4)-(5).  Main  problem  here  is  to  get 
an  accurate  enough  estimate  for  the  sea  clutter  covariance  matrix 
Rd-  Since  the  dimension  of  this  matrix  (prediction  filter)  should 
be  significantly  greater  than  the  number  of  missing  sweeps  -  real 
(corrupted)  data  should  not  be  used  directly  for  sample  matrix  es¬ 
timation  (1)  in  the  way  it  has  been  done  in  our  previous  study 
with  uncorrupted  data.  Since  all  ranges  are  usually  equally  cor¬ 
rupted  by  impulsive  noise,  spatial  diversity  could  be  explored  to 
assist  sea-clutter  covariance  matrix  estimation.  Indeed,  in  most 
cases  truly  dominant  impulsive  noise  sources  are  well  localized 
and  even  with  conventional  beamformer  it  is  usually  possible  to 
find  the  least  contaminated  direction  (beam).  While  Fig  2  displays 
the  range  map  for  the  most  occupied  beam,  the  top  line  in  Fig  6 
demonstrates  distribution  of  this  impulsive  noise  power-to-noise 
ratio  across  the  beams.  It  is  quite  obvious,  that  in  the  “minimal” 
beam  (N=7)  the  power  of  this  noise  is  significantly  smaller  and 
range  processed  data  of  this  beam  could  be  used  for  covariance 
matrix  estimation.  Obviously,  adaptive  spatial  processing  is  even 
more  effective  in  terms  of  reduction  of  antenna  pattern  sidelobes 
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affected  by  impulsive  noise.  The  bottom  line  in  Fig  6  presents  the 
similar  impulsive  noise  to  white  noise  ratio  as  a  function  of  beam 
direction  for  spatial  adaptive  processing  (SAP).  Here  sea  clutter- 
free  ranges  are  used  to  estimate  sample  spatial  covariance  matrix 

R9p\ 


K  =  Y,  xZx*f  (12) 

dj& 


where 


~  \xd,j>'"ixd>3  ) 


(13) 


9  is  the  sea-clutter  free  ranges  area  and  SAP  beamformer  is  defined 
as  usual  by 


WsAP(l) 


RSpSl 
Sf *  Rsp$1 


(14) 


with  Si  as  a  32-variate  steering  vector. 

The  most  important  issue  that  needs  to  be  addressed  to  jus¬ 
tify  this  approach  is  sensitivity  of  this  technique  with  respect  to 
sea-clutter  azimuthal  variability.  Indeed,  we  are  prepared  to  use 
the  sea  clutter  training  data  collected  at  the  output  of  one  particu¬ 
lar  (adaptive)  beam,  but  apply  it  to  quite  a  different  (conventional) 
beamformer  output.  In  order  to  investigate  the  efficiency  of  such 
technique,  we  analysed  our  “clean”  data  with  nominated  “missing” 
data.  For  Weiner  prediction  filter  there  is  no  visible  difference  for 
one  range-doppler  cut  between  the  original  data  and  restored  ones 
even  for  quite  large  number  of  missing  data  (m=100),  regardless 
of  the  fact  if  the  data  are  random  or  consecutive.  For  the  opti¬ 
mization  filter  the  same  can  be  said  for  random  distribution  of  the 
replaced  sweeps  However  the  same  cannot  be  said  for  consecu¬ 
tive  sweeps  as  the  inverted  matrix  becomes  ill  conditioned.  The 
Fig  4  demostraies.  there  is  some  degradation  of  the  signal  in  the 
“sea  clutter”  area 

Final K  we  illustrate  the  practical  efficiency  of  our  approach 
buy  processing  the  real  contaminated  data  shown  in  Fig  1,2. 

For  the  data  shown  m  Fig  1  optimization  technique  (1 1)  should 
be  appropriate  since  onl>  one  dominant  “strike”  is  recorded  here. 

The  results  of  this  technique  applied  to  one  beam  Fig  5  demon¬ 
strate  quite  impressive  improvement  in  sub-clutter  visibility.  Par¬ 
ticular  range  profile  shown  in  Fig  7  demonstrates  35  dB  improve¬ 
ment  in  sub-clutter  v  tsi hi ht>.  w  hile  the  standard  SAP  has  delivered 
only  mere  S- 10  dB 

Our  second  example  deals  with  the  man-made  impulsive  noise, 
as  per  Fig  2  W  hi lc  SAP  is  practically  not  effective  for  the  data 
collected  at  the  direction  of  impulsive  noise  arrival  (beam  4)  it 
provides  quite  reasonable  improvement  for  beam  7  data  that  thus 
could  be  used  as  training  one  The  above  described  approach  with 
adapuve  prediction  filter  “trained”  on  beam  7  data  and  applied  to 
beam  four  data  is  illustrated  by  Fig  8.  The  particular  range  pro¬ 
file  demonstrates  improvement  in  sub-clutter  visibility  up  to  20 
dB.  Note  that  quite  significant  part  of  coherent  processing  interval 
(CPI)  has  been  contaminated  here.  Interestingly  enough,  when  the 
same  prediction  filter  is  applied  to  the  training  data  of  the  beam 
7  (Fig  9),  still  considerable  additional  improvement  with  respect 
to  the  SAP  processing  has  been  obtained.  The  reason  behind  be¬ 
comes  clear  when  sea  clutter  free  ranges  processed  by  SAP  are 
analysed:  impulsive  noise  residues  are  still  some  10  dB  above  the 
ambient  noise  floor.  Therefore  the  replacement  of  these  corrupted 
repetition  intervals  by  predicted  ones  results  in  additional  improve¬ 
ment  in  sub-clutter  visibility. 


2.  CONCLUSION 

Analysis  of  selected  temporal  and  spatial  adaptive  techniques  for 
atmospheric  and  man-made  impulsive  noise  mitigation  for  SW 
OTHR  has  been  performed.  It  has  been  demonstrated  that  tem¬ 
poral  or  spatial  only  processing  could  be  effective  only  in  special 
cases.  For  contaminated  repetition  periods  which  are  randomly 
distributed  over  CPI,  direct  optimization  method  is  shown  to  be 
very  effective.  For  some  beam  directions  affected  by  impulsive 
noise  via  antenna  pattern  sidelobes,  standard  spatial  adaptive  pro¬ 
cessing  could  be  also  quite  effective  on  its  own.  In  more  general 
case,  when  the  number  of  contaminated  sweeps  is  quite  signifi¬ 
cant  and  consecutive  (man-made  impulsive  noise),  and  impulsive 
noise  must  be  rejected  in  all  directions,  proposed  spatio-temporal 
adaptive  processing  is  shown  to  be  most  effective.  Here  spatial 
(adaptive)  processing  is  used  for  initial  impulsive  noise  mitiga¬ 
tion,  and  the  beam  where  this  reduction  is  maximal  is  used  as  a 
training  one  for  sea-clutter  (temporal)  covariance  matrix  estima¬ 
tion.  Adaptive  Weiner  filter  trained  by  the  spatially  processed  data 
is  then  applied  to  contaminated  (conventionally)  beamformed  data 
with  similar  energetic  content  of  Doppler  spectra. 
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Figure  1:  Atmospheric  impulsive  noise,  beam  1. 


Figure  2:  Man-made  impulsive  noise,  beam  4. 
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Figure  4:  Comparison  of  original  data  and  optimization  filter  with  100  consecutive  sweeps  replaced. 


Figure  5:  Comparison  of  conventional,  SAP  and  optimization  beamformer. 
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Figure  9:  Weiner  prediction  filter  trainde  on  beam  7,  used  on  beam  7  top  line  -  SAP  bottom  line  -  Weiner  prediction  filter. 
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